Improving frameless rendering by focusing on change (Online ID 0319)

Realtime rendering requires accurate display of a dynamic scene with minimal delay. Frameless rendering [Bishop et al. 1994] offers unique flexibility in this regard: because it samples time per pixel, it can respond to change with very little delay, and at any location in the image. However, sampling is random, resulting in blurring in changing image regions. We present an approach for improving frameless rendering by making sampling sensitive to change in the image, as suggested in [Bishop et al. 1994]. By measuring this change in visual terms, we are able to direct sampling to those regions of change. The resulting algorithm produces sharper imagery, while introducing minimal overhead into the standard frameless algorithm. We measure change by monitoring color differences in the image, using the summed squared difference between component colors at a pixel in the previous and current rendering. We use a probability distribution function (PDF) to choose the next pixel rendered so that changing image regions are sampled more frequently. The probability of every pixel is the weighted sum of its color difference and its age (time since it was last updated) both normalized over the entire image. The former biases rendering toward regions of change and the latter monitors for change in previously static image regions. This ensures that all pixels are sampled with a certain minimal frequency. The PDF is subsampled into rectangular tiles in image space. Besides bringing obvious improvements in speed, subsampling implements a spatially coherent response to change: if one pixel is changing, neighboring pixels are likely also changing. The probability that one of the pixels in a tile will be rendered is computed using the summed probabilities of the component pixels, normalized by the summed probabilities of all pixels in the image. In order to determine which pixel to render, we first select the tile according to the subsampled PDF, and then randomly select a pixel within that tile with bilinear interpolation of the surrounding tile probabilities just as in bilinear texture filtering. Our renderer displays sharper imagery while using the same number of rays as a conventional frameless renderer. Figures 1 and 2 show corresponding frames of a video at a simulated rendering rate of 900,000 rays per second. In interactive use, our current renderer casts roughly two thirds as many rays per second as the standard renderer, but the resulting images are still sharper. Our major goals in future research will be improving dynamic image quality, evaluating and tuning the performance vs. accuracy tradeoff, and comparing this approach to existing approaches. To improve image quality, we will investigate alternative image quality metrics, including sensitivity to spatial contrast and Gibsonian patterns of motion. We will also incorporate progressive rendering [Bergman et al. 1986] into the ray tracer, enabling control of the tradeoff between temporal and spatial accuracy – a line of research that we believe will be particularly fruitful. In evaluation and comparison, our standard will be a recorded animation that simulates the performance of a hypothetical instantaneous renderer, one that can produce frames in response to input without any delay. We can then compare the performance of our (or any) algorithm to the output of this hypothetically ideal renderer moment by moment. Ultimately, we will examine the use of our change sensitive approach in the context of distributed rendering [Parker et al. 1999].